Large Language Models (LLMs) have demonstrated significant potential in decision-making and reasoning, particularly when integrated with various tools to effectively solve complex problems. However, existing benchmarks for evaluating LLMs' tool usage face several limitations: (1) limited evaluation scenarios, often lacking assessments in real multi-turn dialogue contexts; (2) narrow evaluation dimensions, with insufficient detailed assessments of how LLMs use tools; and (3) reliance on LLMs or real API executions for evaluation, which introduces significant overhead. To address these challenges, we introduce ACEBench, a comprehensive benchmark for assessing tool usage in LLMs. ACEBench categorizes data into three primary types based on evaluation methodology: Normal, Special, and Agent. "Normal" evaluates tool usage in basic scenarios; "Special" evaluates tool usage in situations with ambiguous or incomplete instructions; "Agent" evaluates tool usage through multi-agent interactions to simulate real-world, multi-turn dialogues. We conducted extensive experiments using ACEBench, analyzing various LLMs in-depth and providing a more granular examination of error causes across different data types.

IBM has once again partnered with the United States Tennis Association (USTA) and they're using new AI-powered tools during the US Open to deliver AI-generated highlights, real-time stats and match analysis, as well as an onsite experience center where attendees can experience AI in action. There's a new technology solution, IBM Coach Advisor, that uses AI and analytics to quantify a player's physical exertion and endurance and make correlations to match performance, both during regular games and at the US Open. There's also IBM Watson OpenScale, which figures out the most emotion-packed moments on court for highlight reels. This means that fans watching the men's finals with Danlil Medvedev and Rafael Nadal, or watching highlights from the women's finals with Serena Williams and Bianca Andreescu will be able to view the most breath-taking moments thanks to AI. This is different from the traditional way that coaches assess an athlete's mechanics and endurance.

Australian Open champion Naomi Osaka says she had to be a "robot" and turn off her feelings to hold her nerve and win the final against Petra Kvitova. The Japanese, 21, had tears in her eyes after having three match points saved by her Czech opponent in the second set - before winning 7-6 (7-2) 5-7 6-4. "You know how some people get worked up about things? That's a very human thing to do," said Osaka. "Sometimes I feel like I don't want to waste my energy doing stuff like that."